Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation

This paper studies the robustness of reinforcement learning algorithms to errors in process. Specifically, we revisit benchmark problem discrete-time linear quadratic regulation (LQR) and study long-standing open question: Under what conditions is policy iteration method robustly stable from a dynamical systems perspective? Using advanced stability results control theory, it shown that for LQR inherently robust small process enjoys small-disturbance input-to-state stability: whenever error each bounded small, solutions algorithm are also bounded, and, moreover, enter stay neighborhood optimal solution. As an application, novel off-policy optimistic least-squares proposed, when system dynamics subjected additive stochastic disturbances. The proposed new validated by numerical example.